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(54) Method for recognizing speech on a mobile terminal 

(57) When a cellular phone user presses a key and 
a controller 23 detects that the key has been pressed, 
the controller 23 outputs a speech-recognition-start 
sound. At the same time, the controller 23 causes a 
timer circuit 23A to start measuring a predetermined 
time of t1. When the predetermined time of t1 elapses 
after the speech-recognition-start sound is generated, 
the controller 23 starts a speech recognition operation 
and detects a user's speech. At this time, the controller 
23 causes the timer circuit 23A to start measuring a pre- 
determined time of t2. When the predetermined time of 
t2 elapses, the controller 23 stops the sound. After that, 
when a speech recognizing unit 25 detects the speech, 
the controller 23 controls the speech recognizing unit 25 
to perform speech recognition processing and deter- 
mines what is the detected speech. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001 ] The present invention relates to a mobile termi- 
nal and more particularly to a mobile terminal having the 
speech recognition function. 

Description pf the Related Art 

[0002] Conventionally, a mobile terminal having the 
speech recognition function such as a cellular phone 
performs various control functions and operations: for 
example, it recognizes spoken sounds, translates them 
from one language to another, or outputs dial signals. 
[0003] In this type of cellular phone, it is possible to 
recognize speech for a predetermined period only when 
the user presses a key on the phone. In this case, 
before starting recognizing speech, it is necessary to 
inform the user of the start of recognizing speech. More 
specifically, this process is done as follows. When the 
user presses the speech recognition start key, the 
phone emits a start sound for a predetermined period to 
inform the user that the phone will start speech recogni- 
tion. After the predetermined time elapses, the phone 
stops the start sound and starts the speech recognition 
operation. Then, upon detection of user's speech, the 
phone recognizes it according to the predetermined 
method. 

[0004] However, as the user becomes familiar with this 
speech recognition start operation, he or she some- 
times starts speaking before the speech-recognition- 
start sound stops. This prevents the start of a speech, 
the most important part of speech recognition, from 
being recognized correctly, sometimes resulting in the 
whole speech being recognized incorrectly 
[0005] On the other hand, on a non-mobile unit, an 
approach sensor provided near the microphone senses 
the distance between the speaker's face and the micro- 
phone. Such a sensor is disclosed in Japanese laid- 
open patent application heisei 2-131300. Upon sensing 
that the speaker's face comes sufficiently dose to the 
microphone, the unit starts speech recognition, thereaf- 
ter it displays a message indicating that it is ready to 
receive speech. 

SUMMARY OF THE INVENTION 

[0006] It is an object of this invention to provide a 
mobile terminal which has been improved on the con- 
ventional mobile terminal. 

[0007] It is another object of this invention to provide 
a mobile terminal which performs correct speech detec- 
tion, the important part of speech recognition, to prevent 
speech from being recognized incorrectly. 
[0008] To achieve the above objects, the mobile termi- 



nal according to this invention includes a notification 
unit notifying a user of a start of speech recognition for 
a predetermined time, a speech recognizing unit recog- 
nizing speech and a start control unit starting speech 

5 recognition according to the speech recognizing unit 
before notification stops. The mobile terminal according 
to this invention further includes a storage unit storing a 
correspondence between partner's names and tele- 
phone numbers, an extraction unit extracting, when the 

w speech recognizing unit recognizes the partner's name, 
the telephone number corresponding to the partner's 
name stored in the storage unit and an automatic calling 
unit making an automatic call to the telephone number 
extracted by the extraction unit. Preferably, the start 

15 control unit starts speech recognition according to the 
speech recognizing unit during a start of the notification 
and an end of it. Preferably, the mobile terminal accord- 
ing to this invention further includes a first measuring 
unit for measuring a first predetermined time after the 

20 start of the notification and the start control unit starts 
speech recognition according to the speech recognizing 
unit after the first predetermined time elapses. Prefera- 
bly, the mobile terminal according to this invention fur- 
ther includes a second measuring unit measuring a 

25 second predetermined time after the start of the speech 
recognition; and a stop control unit stopping the notifica- 
tion after the second predetermined time elapses. It is 
preferable that the notification is started by a user's 
operation, that the operation is a key operation, and that 

30 the notifying unit notifies the start of speech recognition 
through a sound. The notifying unit may notify the start 
of speech recognition through at least one of a sound, a 
vibrator, and a LED. 

[0009] The speech recognition method for use on the 

35 mobile terminal according to this invention includes the 
steps of notifying for a predetermined time that the ter- 
minal will start speech recognition; and starting the 
speech recognition before the notification stops. The 
speech recognition method for use on the mobile termi- 

40 nal according to this invention further includes the steps 
of storing a correspondence between partner's names 
and telephone numbers; recognizing the partner's 
name through the speech recognition; extracting the tel- 
ephone number corresponding to the partner's name, 

45 the partner's name and the telephone number being 
stored in the storing step; and making an automatic call 
to the telephone number extracted by the extraction 
step. Preferably, the speech recognition start step starts 
said speech recognition at a point in time during a start 

so of the notification and an end of it. Preferably, the 
speech recognition method for use on the mobile termi- 
nal further includes the step of measuring a first prede- 
termined time after the start of the notification and the 
speech recognition start step starts recognizing speech 

55 after the first predetermined time elapses. The speech 
recognition method for use on the mobile terminal 
according to this invention further includes the steps of 
measuring a second predetermined time after the start 
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of the speech recognition; and stopping the notification 
after the second predetermined time elapses. It is pref- 
erable that the notification is started by a user's opera- 
tion, that the operation is a key operation, and that the 
notifying step notifies the start of speech recognition 
through a sound. The notifying step may notify the start 
of speech recognition through at least one of a sound, a 
vibrator, and a LED. 

[001 0] Thus, the terminal or the method according to 
this invention starts the speech recognition operation 
(speech detection) while the speech-recognition-start 
sound is being emitted. This allows the speech recogni- 
tion function to correctly detect user's speech even if it 
is given before the speech-recognition-start sound 
stops, preventing the speech from being recognized 
incorrectly. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[001 1 ] These and other objects, features and advan- 
tages of the invention will become more fully apparent 
from the following detailed description taken in conjunc- 
tion with accompanying drawings. 

Figs. 1a - 1d are timing diagrams showing the 
speech recognition operation of a phone which 
starts speech recognition as instructed by a user. 
Fig. 2 is a flowchart showing the speech recognition 
operation of the telephone which starts speech rec- 
ognition as instructed by the user. 
Fig. 3 is a diagram showing the preferable circuit 
configuration of an embodiment of a cellular phone 
according to this invention. 
Fig. 4 is a flowchart showing the preferable opera- 
tion of the embodiment of the cellular phone shown 
in Fig. 3. 

Figs. 5a - 5d are preferable timing diagrams show- 
ing the speech recognition operation of the embod- 
iment of the cellular phone shown in Fig. 3. 
Fig. 6 is a preferable diagram showing the internal 
structure of the RAM of the embodiment shown in 
Fig. 3. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

[0012] Referring now to Figs. 1a - 1d and Fig. 2 for a 
more complete understanding of this invention, the fol- 
lowing describes the speech recognition function of a 
mobile terminal which starts speech recognition as 
instructed by a user. 

[001 3] When the user presses a speech recognition 
start key as shown in Fig. 1a (step 51 in Fig. 2), the ter- 
minal gives a speech-recognition-start sound informing 
the user of the start of speech recognition as shown in 
Fig. 1c (steps 52 and 53 in Fig. 2). After T seconds, the 
terminal stops the speech-recognition-start sound and, 
as shown in Fig. 1d, starts the speech recognition oper- 



ation (step 54 in Fig. 2). Upon detecting user's speech 
(step 55 in Fig. 2), the terminal recognizes the detected 
speech according to a predetermined method as shown 
in Fig. 1b (step 56 in Fig. 2). 

5 [001 4] However, as the user becomes familiar with this 
speech recognition start operation on this mobile termi- 
nal, he or she sometimes starts speaking before the 
start sound stops. In this case, user's speech given 
before speech recognition starts is not detected. This 

10 prevents the start of the speech, the most important 
part of speech recognition, from being recognized cor- 
rectly, sometimes resulting in the whole speech being 
recognized incorrectly. 

[0015] To prevent the speech from being recognized 
15 incorrectly, the terminal according to this invention 
starts the speech recognition operation (speech detec- 
tion) while the speech-recognition-start sound is being 
emitted. This allows the speech recognition function to 
correctly detect user's speech even rf it is given before 
20 the speech-recognition-start sound stops. 

[001 6] Referring to Figs. 3 to 6, the following describes 
this invention more in detail. Fig. 3 is a diagram showing 
the circuit of a preferable embodiment of a mobile termi- 
nal, such as a cellular phone, according to this inven- 
25 tion. 

[001 7] A radio unit 22 shown in Fig. 3 frequency-con- 
verts and demodulates radio waves received via an 
antenna 21. The radio unit 22 also modulates and fre- 
quency-converts transmission signals received from a 

30 controller 23. A display 24 displays the functions of the 
cellular phone or the communication status of it. A 
speech recognizing unit 25 detects and recognizes 
user-generated speech entered from a microphone 27. 
A key input section 26 is used when the user operates 

35 keys on the cellular phone. A speaker 28 outputs, during 
communication, partner's speech or sounds generated 
by the controller 23. The controller 23 controls the radio 
unit 22 to process send/receive data, the speech recog- 
nizing unit 25 to do the speech recognition operation, 

40 the display 24 to display the speech recognition results 
generated by the speech recognizing unit 25, and the 
key input section 26 to process on the basis of the oper- 
ated key. The controller 23 contains a timer circuit 23A. 
This circuit 23A controls the time at which the speech- 

45 recognition-start sound is generated or the duration 
from the time the speech recognition operation starts to 
the time the sound stops. A RAM 29 stores partner's 
names and their telephone numbers corresponding to it 
displayed on the display 24 when the telephone direc- 

so tory function is used. 

[0018] When the user performs operation via the key 
input section 26 (for example, presses the send button) 
and speaks to the microphone 27, the speech recogniz- 
ing unit 25 recognizes the spoken speech (for example, 

55 a partner's name). The unit sends the result to the con- 
troller 23 for analysis. Based on the analyzed result, the 
controller 23 performs the corresponding operation; for 
example, it checks data stored in the RAM 29 and auto- 



3 



5 



EP0 939 534A1 



6 



matically dials the telephone number corresponding to 
the partner's name. 

[001 9] Fig. 4 is a flowchart showing the operation of a 
speech recognition unit in the cellular phone used in the 
embodiment. Figs. 5a to 5d are timing diagrams show- 
ing the speech recognition operation of the speech rec- 
ognition unit in the cellular phone used in the 
embodiment. 

[0020] As shown in Fig. 5a, when the cellular phone 
user presses a key and the controller 23 detects that the 
key has been pressed (step 31 in Fig. 4), the controller 
23 outputs the speech-recognition-start sound as 
shown in Rg. 5c to inform the user that the unit is going 
to start speech recognition (step 32 in Fig. 4). The key 
operation in step 31 in Fig. 4 triggers the speech entry. 
Any button - send button, dial button, or special trigger 
button - may be used for this operation as long as it 
does not affect normal key operation. Then, the control- 
ler 23 causes the timer circuit 23A to start measuring a 
predetermined time of t1, for example, 200ms (step 33 
in Fig. 4). When the predetermined time of t1 elapses 
after the speech-recognition-start sound is generated 
(step 34 in Fig. 4), the controller 23 starts the speech 
recognition operation as shown in Fig. 5d and detects 
the user's speech as shown in Fig. 5b. At this time, the 
controller 23 also causes the timer circuit 23 A to start 
measuring a predetermined time of t2, for example 
100ms (step 35 in Fig. 4). When the predetermined time 
of t2 elapses (step 36 in Rg. 4), the controller 23 stops 
the sound (step 37 in Fig. 4). After that, when the 
speech recognizing unit 25 detects the Speech (step 38 
in Fig. 4), the controller 23 controls the speech recog- 
nizing unit 25 to perform speech recognition processing 
and determines what is the detected speech (step 39 in 
Fig. 4). The controller 23 processes the result of speech 
recognition and controls the cellular phone according to 
a predetermined procedure. For example, it displays the 
result of speech recognition on the display 24 or emits 
sounds from the speaker 28. 
[0021 ] Fig. 6 is a diagram showing the internal struc- 
ture of the RAM 29 used in the preferred embodiment 
shown in Fig. 3. As shown in Fig. 6, the RAM 29 stores 
partner's names and their telephone numbers corre- 
sponding them. For example, when the controller 23 
detects "Mary" as a result of speech recognition in step 
39 in Fig. 4, it extracts the telephone number "010-123- 
4567" corresponding to "Mary". After that, a call to the 
telephone number "101-123-4567" is made automati- 
cally. 

[0022] When the predetermined time of t1 has 
elapsed after the speech-recognition-start sound is 
generated, the cellular phone used in the embodiment 
starts the speech recognition operation (speech detec- 
tion). And, after the predetermined time of t2 elapses 
from the start of the speech recognition operation, the 
speech-recognition-start sound stops. That is, the cellu- 
lar phone starts the speech recognition operation 
(speech detection) before the speech-recognition-start 



sound stops. Therefore, even rf the user starts speaking 
even before the speech-recognition-start sound stops, 
the cellular phone is able to do the speech recognition 
operation (speech detection) during the predetermined 

5 time of t2. 

[0023] The preferred embodiment of this invention has 
been described. However, the mobile terminal accord- 
ing to this invention is not limited to the above embodi- 
ment. For example, the cellular phone used in this 

10 embodiment outputs the speech-recognition-start 
sound to inform the user of the start of speech recogni- 
tion. Instead of this speech-recognition-start sound, it is 
possible to use anything else that tells the user that the 
cellular phone starts speech recognition. For example, 

is the light of a light emitting diode, a vibrator, a character 
or a picture on the display, or a synthesized sound may 
be used. In addition, the speech-recognition-start sound 
may be output not only when the user presses a key but 
also after a special message is displayed. 

20 [0024] In the above embodiment, a cellular phone is 
used as an example of the mobile terminal according to 
this invention. This mobile terminal may be any unit with 
the speech recognition function. For example, a mobile 
radio unit such as a pager, a mobile data terminal with 

25 the cellular phone function, a wire communication unit, 
or a personal computer may be used as long as they 
have the speech recognition function. 
[0025] As described above, the speech recognition 
unit according to this invention starts the speech recog- 

30 nition operation (speech detection) before the speech- 
recognition-start sound stops. So, even if the user starts 
speaking before the speech-recognition-start sound 
stops, the unit detects the start of a speech, the most 
important part of speech recognition. This prevents the 

35 speech from being recognized incorrectly. 

[0026] Obviously, numerous additional modifications 
and variations of the present invention are possible in 
light of the above teachings, ft is, therefore, to be under- 
stood that within the scope of the appended claims, the 

40 invention may be practiced otherwise than as specifi- 
cally described herein. 

Claims 

45 1. A mobile terminal comprising: 

notifying means for notifying a user of a start of 
speech recognition for a predetermined time; 
speech recognizing means for recognizing 
so speech; and 

start control means for starting speech recogni- 
tion according to said speech recognizing 
means before notification stops. 

55 2. The mobile terminal as claimed in claim 1, further 
comprising: 

storage means for storing a correspondence 
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between partner's names and telephone num- 
bers; 

extraction means for extracting, when said 
speech recognizing means recognizes said 
partner's name, the telephone number corre- 5 
sponding to said partner's name stored in said 
storage means; and 

automatic calling means for making an auto- 
matic call to the telephone number extracted by 
said extraction means. 10 

3. The mobile terminal as claimed in claim 1 , wherein 
said start control means starts speech recognition 
according to said speech recognizing means during 

a start of said notification and an end of it 15 

4. The mobile terminal as claimed in claim 3, further 
comprising: 

first measuring means for measuring a first pre- 20 
determined time after a start of said notification 
and wherein said start control means starts 
speech recognition according to said speech 
recognizing means after said first predeter- 
mined time elapses. 25 

5. The mobile terminal as claimed in claim 4 further 
comprising: 

second measuring means for measuring a sec- 30 
ond predetermined time after the start of said 
speech recognition; and 

stop control means for stopping said notifica- 
tion after said second predetermined time 35 
elapses. 

6. The mobile terminal as claimed in claim 1 , wherein 
said notification is started by a user's operation. 

40 

7. The mobile terminal as claimed in claim 6, wherein 
said operation is a key operation. 

8. The mobile terminal as claimed in claim 1 , wherein 
said notifying means notifies a start of speech rec- 45 
ognition through a sound. 

9. The mobile terminal as claimed in claim 1 , wherein 
said notifying means notifies a start of speech rec- 
ognition through at least one of a sound, a vibrator, so 
and a LED. 

10. A speech recognition method for use on a mobile 
terminal comprising the steps of: 

55 

notifying for a predetermined time that said ter- 
minal will start speech recognition; and 
starting said speech recognition before said 



notification stops. 

11. The speech recognition method for use on the 
mobile terminal as claimed in claim 10, further com- 
prising the steps of: 

storing a correspondence between partner's 
names and telephone numbers; 
recognizing said partner's name through said 
speech recognition; 

extracting the telephone number correspond- 
ing to said partner's name, the partner's name 
and the telephone number being stored in said 
storing step; and 

making an automatic call to the telephone 
number extracted by said extraction step. 

12. TTie speech recognition method for use on the 
mobile terminal as claimed in claim 10, wherein 
said speech recognition start step starts said 
speech recognition at a point in time during a start 
of said notification and an end of it. 

13. The speech recognition method for use on the 
mobile terminal as claimed in claim 10, further com- 
prising the step of: 

measuring a first predetermined time after a 
start of said notification and wherein said 
speech recognition start step starts recogniz- 
ing speech after said first predetermined time 
elapses. 

14. The speech recognition method for use on the 
mobile terminal as claimed in claim 13, further com- 
prising the steps of: 

measuring a second predetermined time after 
the start of said speech recognition; and 
stopping said notification after said second pre- 
determined time elapses. 

15. The speech recognition method for use on the 
mobile terminal as claimed in claim 10, wherein 
said notification is started by a user's operation. 

16. The speech recognition method for use on the 
mobile terminal as claimed in claim 15, wherein 
said operation is a key operation. 

17. The speech recognition method for use on the 
mobile terminal as claimed in claim 10, wherein 
said notifying step notifies a start of speech recog- 
nition through a sound. 

18. The speech recognition method for use on the 
mobile terminal as claimed in claim 10, wherein 
said notifying step notifies a start of speech recog- 
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and a LED. 
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